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About the book 


¢ This version was published on Dec 12 2020 


This is an open-source introduction to Bash scripting guide that will help 
you learn the basics of Bash scripting and start writing awesome Bash 
scripts that will help you automate your daily SysOps, DevOps, and Dev 
tasks. No matter if you are a DevOps/SysOps engineer, developer, or 
just a Linux enthusiast, you can use Bash scripts to combine different 
Linux commands and automate tedious and repetitive daily tasks so 
that you can focus on more productive and fun things. 


The guide is suitable for anyone working as a developer, system 
administrator, or a DevOps engineer and wants to learn the basics of 
Bash scripting. 


The first 13 chapters would be purely focused on getting some solid 
Bash scripting foundations, then the rest of the chapters would give you 
some real-life examples and scripts. 





About the author 


My name is Bobby Iliev, and | have been working as a Linux DevOps 
Engineer since 2014. | am an avid Linux lover and supporter of the 
open-source movement philosophy. | am always doing that which | 
cannot do in order that | may learn how to do it, and | believe in sharing 
knowledge. 


| think it's essential always to keep professional and surround yourself 
with good people, work hard, and be nice to everyone. You have to 
perform at a consistently higher level than others. That's the mark of a 
true professional. 


For more information, please visit my blog at https://bobbyiliev.com, 
follow me on Twitter @bobbyiliev_ and YouTube. 





Sponsors 


This book is made possible thanks to these fantastic companies! 


DigitalOcean 


DigitalOcean is a cloud services platform delivering the simplicity 
developers love and businesses trust to run production applications at 
scale. 


It provides highly available, secure, and scalable compute, storage, and 
networking solutions that help developers build great software faster. 


Founded in 2012 with offices in New York and Cambridge, MA, 
DigitalOcean offers transparent and affordable pricing, an elegant user 
interface, and one of the largest libraries of open source resources 
available. 


For more information, please visit https://www.digitalocean.com or 


follow @digitalocean on Twitter. 


If you are new to DigitalOcean, you can get a free $100 credit and spin 
up your own servers via this referral link here: 


Free $100 Credit For DigitalOcean 


DevDojo 


The DevDojo is a resource to learn all things web development and web 
design. Learn on your lunch break or wake up and enjoy a cup of coffee 
with us to learn something new. 


Join this developer community, and we can all learn together, build 
together, and grow together. 





loin DevDojo 


For more information, please visit https://www.devdojo.com or follow 
@thedevdojo on Twitter. 
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This ebook was generated by Ibis developed by Mohamed Said. 


Ibis is a PHP tool that helps you write eBooks in markdown. 
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Introduction to Bash scripting 


Welcome to this Bash basics training guide! In this bash crash course, 
you will learn the Bash basics so you could start writing your own Bash 
scripts and automate your daily tasks. 


Bash is a Unix shell and command language. It is widely available on 
various operating systems, and it is also the default command 
interpreter on most Linux systems. 


Bash stands for Bourne-Again SHell. As with other shells, you can use 
Bash interactively directly in your terminal, and also, you can use Bash 
like any other programming language to write scripts. This book will 
help you learn the basics of Bash scripting including Bash Variables, 
User Input, Comments, Arguments, Arrays, Conditional Expressions, 
Conditionals, Loops, Functions, Debugging, and testing. 


In order to write Bash scripts, you just need a UNIX terminal and a text 
editor like Sublime Text, VS Code, or a terminal-based editor like vim or 
nano. 
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Bash Structure 


Let's start by creating a new file with a .sh extension. As an example, 
we could create a file called devdojo. sh. 


To create that file, you can use the touch command: 


| touch devdojo.sh 


Or you can use your text editor instead: 


| nano devdojo.sh 


In order to execute/run a bash script file with the bash shell interpreter, 
the first line of a script file must indicate the absolute path to the bash 
executable: 


| #!/bin/bash 


This is also called a Shebang. 


All that the shebang does is to instruct the operating system to run the 
Script with the /bin/bash executable. 


ie 





SPL iMa Cs OCemM ovale! 


Once we have our devdojo.sh file created and we've specified the bash 
shebang on the very first line, we are ready to create our first Hello 
World bash script. 


To do that, open the devdojo.sh file again and add the following after 
the #!/bin/bash line: 


#!/bin/bash 


echo "Hello World!" 


Save the file and exit. 


After that make the script executable by running: 


| chmod +x devdojo.sh 


After that execute the file: 


| ./devdojo.sh 


You will see a "Hello World" message on the screen. 


Another way to run the script would be: 


| bash devdojo.sh 
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As bash can be used interactively, you could run the following 
command directly in your terminal and you would get the same result: 


echo "Hello DevDojo!" 


Putting a script together is useful once you have to combine multiple 
commands together. 


ils) 





Belardes 


As with any other programming language, you can use variables in 
Bash as well. However, there are no data types, and a variable in Bash 
can contain numbers and characters. 


To assign a value to a variable, all you need to do is use the = sign: 


name="DevDojo" 


Notice: as an important note, you can not have spaces before 
el alem= AK) nual see (e 1B 


After that, to access the variable, you have to use the $ and reference it 
like this: 


| echo $name 


Wrapping the variable name between curly brackets is not required but 
is considered good practice, and | would advise to use them whenever 
you can: 


| echo ${name} 


The above would output: DevDojo as this is the value of our variable. 


Next, let's update our devdojo.sh script and include a variable. 
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Again, with your favorite text editor, open the file: 


nano devdojo.sh 


And update the file, so it looks like this: 


#!/bin/bash 
name="DevDojo" 


echo "Hi there $name" 


Save it and run it again: 


| ./devdojo.sh 


You would see the following output on your screen: 


| Hi there DevDojo 


Here is a rundown of the script: 


¢ #!/bin/bash - first, we specified our shebang 

* name=DevDojo - then we defined a variable called name and 
assigned a value to it 

*echo "Hi there $name" - finally we output the content of the 
variable on the screen by using echo 


You can also add multiple variables: 
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#!/bin/bash 


name="DevDojo" 
greeting="Hello" 


echo "$greeting $name" 


SY-hVioml m=] ale mae la mire lel- 110 


| ./devdojo.sh 


You would see the following output on your screen: 


| Hello DevDojo 


Note that you don't necessarily need to add semicolon ; at the end of 
each line. It would work both ways, a bit like in JavaScript! 
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Bash User Input 


With the previous script, we defined a variable, and we output the value 
of the variable on the screen with the echo $name. 


Now let's go ahead and ask the user for input instead. To do that again, 
open the file with your favorite text editor and update the script as 
follows: 


#!/bin/bash 


echo "What is your name?" 
iastolo mm are NICs 


echo "Hi there $name" 
echo "Welcome to DevDojo!" 


The above will prompt the user for input and then store that input as a 
String/text in a variable. 


We can then use the variable and print a message back to them. 
The output of the above script would be: 


¢ First run the script: 


./devdojo.sh 


¢ Then, you would be prompted to enter your name: 


I) 





What is your name? 
Bobby 


¢ Once you've typed your name, just hit enter, and you will get the 
following output: 


Hi there Bobby 
Welcome to DevDojo! 


To reduce the code, we could change the first echo statement with the 
read -p, the read command used with -p flag will print a message 
before prompting the user for their input: 

#!/bin/bash 

read -p "What is your name? " name 


echo "Hi there $name" 
echo "Welcome to DevDojo!" 


Make sure to test this out yourself as well! 
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Bash Comments 


As with any other programming language, you can add comments to 
your script. Comments are used to leave yourself notes through your 
(olel=e 


To do that in Bash, you need to add the # symbol at the beginning of 
the line. Comments will never be rendered on the screen. 


Here is an example of a comment: 


# This is a comment and will not be rendered on the screen 


Let's go ahead and add some comments to our script: 


#!/bin/bash 

# Ask the user for their name 

read -p "What is your name? " name 
# Greet the user 


echo "Hi there $name" 
echo "Welcome to DevDojo!" 


Comments are a great way to describe some of the more complex 
functionality directly in your scripts so that other people could find their 
way around your code with ease. 
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Bash Arguments 


You can pass arguments to your shell script when you execute it. To 
pass an argument, you just need to write it right after the name of your 
Script. For example: 


./devdojo.com your argument 


In the script, we can then use $1 in order to reference the first 
argument that we specified. 


If we pass a second argument, it would be available as $2 and so on. 


Let's create a short script called arguments.sh as an example: 
#!/bin/bash 
echo "Argument one is $1" 


echo "Argument two is $2" 
echo "Argument three is $3" 


Save the file and make it executable: 


| chmod +x arguments.sh 


Then run the file and pass 3 arguments: 


| ./arguments.sh dog cat bird 
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The output that you would get would be: 


Argument one is dog 
Argument two is cat 
Argument three is bird 


To reference all arguments, you can use $@: 


#!/bin/bash 


echo "All arguments: $@" 


If you run the script again: 


| ./arguments.sh dog cat bird 


You will get the following output: 


| ALL arguments: dog cat bird 
Another thing that you need to keep in mind is that $0 is used to 
reference the script itself. 


This is an excellent way to create self destruct the file if you need to or 
just get the name of the script. 


For example, let's create a script that prints out the name of the file and 
deletes the file after that: 
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#!/bin/bash 


echo "The name of the file is: $0 and it is going to be self- 
deleted." 


rm -f $0 


You need to be careful with the self deletion and ensure that you have 
your script backed up before you self-delete it. 
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Bash Arrays 


If you have ever done any programming, you are probably already 
familiar with arrays. 


But just in case you are not a developer, the main thing that you need 
to know is that unlike variables, arrays can hold several values under 
one name. 


You can initialize an array by assigning values devided by space and 
enclosed in (). Example: 


my array=("value 1" "value 2" "value 3" "value 4") 


To access the elements in the array, you need to reference them by 
their numeric index. 


Notice: keep in mind that you need to use curly brackets. 
¢ Access a single element, this would output: value 2 
| echo ${my_ array[1]} 


e This would return the last element: value 4 


| echo ${my_array[-1]} 


ZA5) 





¢ As with command line arguments using @ will return all arguments 
in the array, as follows: value 1 value 2 value 3 value 4 


| echo ${my_ array[@]} 


¢ Prepending the array with a hash sign (#) would output the total 
number of elements in the array, in our case it is 4: 


| echo ${#my array[@]} 


Make sure to test this and practice it at your end with different values. 


26 





Bash Conditional Expressions 


In computer science, conditional statements, conditional expressions, 
and conditional constructs are features of a programming language, 
which perform different computations or actions depending on whether 
a programmer-specified boolean condition evaluates to true or false. 


In Bash, conditional expressions are used by the [[ compound 
command and the [built-in commands to test file attributes and 
perform string and arithmetic comparisons. 


Here is a list of the most popular Bash conditional expressions. You do 
not have to memorize them by heart. You can simply refer back to this 
list whenever you need it! 
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File expressions 


a alm mal (sm). 4iSiace 


| Dee Ai EE) af 


¢ True if file exists and is a block special file. 


| ho) tee 


¢ True if file exists and is a character special file. 


| Ween) 


aa alo Com). 4(S1uoee~] ae Sere me fe Lelne) aye 


| EE teh eel) eb 


¢ True if file exists. 


| Eee ie) 8p 


¢ True if file exists and is a regular file. 


| Lisa ana 


¢ True if file exists and is a symbolic link. 
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| ee) 


e True if file exists and is readable. 


| Eek ee Sy TE 


¢ True if file exists and has a size greater than zero. 


| es a pee eh 


¢ True if file exists and is writable. 


| [{ -w ${file} ]] 


e True if file exists and is executable. 


| ee ie) tp 


¢ True if file exists and is a symbolic link. 


| Dae ee ede 1H 
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String expressions 


¢ True if the shell variable varname is set (has been assigned a 
Vie ]LUL=) B 


| a Sate 

True if the length of the string is zero. 

| [[ -z ${string} ]] 

True If the length of the string is non-zero. 
| [[ -n ${string} ]] 


¢ True if the strings are equal. = should be used with the test 
command for POSIX conformance. When used with the [[ 
command, this performs pattern matching as described above 
(Compound Commands). 


| [{ ${stringl} == ${string2} ]] 


¢ True if the strings are not equal. 


| [{ ${stringl} != ${string2} ]] 


¢ True if stringl sorts before string2 lexicographically. 
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| Less Saad ot a go fl ee et aos ol gas ae at ne 


¢ True if string1 sorts after string2 lexicographically. 


| [{ ${stringl} > ${string2} ]] 
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Arithmetic operators 


¢ Returns true if the numbers are equal 


| I el ete Wb RR SK To fees We ete Pai oat 


¢ Returns true if the numbers are not equal 


| Ess eat 0s Seer (cl ef 2h A 


¢ Returns true if argl1 is less than arg2 


| UES Re ke ee nee ei ate Pala 


¢ Returns true if arg1 is less than or equal arg2 


| Le Celie Ube Kes Gi reels Pin at 


¢ Returns true if argl is greater than arg2 


| Ee etete he Soha chic ale Pat a ld 


¢ Returns true if argl is greater than or equal arg2 


| [i Stargl} ge starg2} 1] 


As a side note, argl and arg2 may be positive or negative integers. 


As with other programming languages you can use AND & OR conditions: 
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[{[ test_case 1 ]] && [[ test_case 2 ]] # And 
[{ test_case 1 ]] || [[ test_case 2 ]] # Or 
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Bash Conditionals 


In the last section, we covered some of the most popular conditional 
expressions. We can now use them with standard conditional 
statements like if, if-else and switch case statements. 
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If statement 


The format of an if statement in Bash is as follows: 


if [[ some test ]] 
then 

<commands> 
sia 


Here is a quick example which would ask you to enter your name in 
case that you've left it empty: 


#!/bin/bash 

# Bash if statement example 

read -p "What is your name? " name 
if [[ -z ${name} ]] 

ag atoia) 


echo "Please enter your name!" 
ie 
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If Else statement 


With an if-else statement, you can specify an action in case that the 
condition in the if statement does not match. We can combine this 
with the conditional expressions from the previous section as follows: 


#!/bin/bash 
# Bash if statement example 
read -p "What is your name? " name 
if [[ -z ${name} ]] 
then 

echo "Please enter your name!" 
else 


echo "Hi there ${name}" 
fi 


You can use the above if statement with all of the conditional 
expressions from the previous chapters: 


#!/bin/bash 

admin="devdojo" 

read -p "Enter your username? " username 

# Check if the username provided is the admin 

if [[ "${username}" == "${admin}" ]] ; then 
echo "You are the admin user!" 

else 


echo "You are NOT the admin user!" 
fe 


Here is another example of an if statement which would check your 
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current User ID and would not allow you to run the script as the root 
user: 


#!/bin/bash 


if ({SEUID == 0: ).) + then 
echo "Please do not run as root" 
exit 

ries 


If you put this on top of your script it would exit in case that the EUID is 
0 and would not execute the rest of the script. This was discussed on 


the DigitalOcean community forum. 
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Switch case statements 


As in other programming languages, you can use a case statement to 
simplify complex conditionals when there are multiple different choices. 
So rather than using a few if, and if-else statements, you could use a 
Single case statement. 


The Bash case statement syntax looks like this: 


case $some variable in 


pattern 1) 
commands 


ad 


pattern _2| pattern_3) 
commands 


ad 


=) 
default commands 


ad 


too 


A quick rundown of the structure: 


¢ All case statements start with the case keyword. 

¢ On the same like as the case keyword, you need to specify a 
variable or an expression followed by the in keyword. 

¢ After that, you have your case patterns, where you need to use ) 
to identify the end of the pattern. 

¢ You can specify multiple patterns divided by a pipe: |. 

¢ After the pattern, you specify the commands that you would like to 
be executed in case that the pattern matches the variable or the 
expression that you've specified. 

¢ All clauses have to be terminated by adding ;; at the end. 
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¢ You can have a default statement by adding a * as the pattern. 
¢ To close the case statement, use the esac (case typed backwards) 
keyword. 


Here is an example of a Bash case statement: 


#!/bin/bash 


echo -n "Enter the name of a car brand: 
read car 


case $car in 


Tesla) 
echo -n "${car}'s factory in the USA." 


ify 


BMW | Mercedes | Audi | Porsche) 
echo -n "${car}'s factory in Germany." 


ft 


Toyoda | Mazda | Mitsubishi | Subaru) 
echo -n "${car}'s factory in Japan." 


Le a 


wi) 
echo -n "${car} is an unknown car brand." 
) 


[Sy l(e 
With this script, we are asking the user to input a name of a car brand 
like Telsa, BMW, Mercedes and etc. 


Then with a case statement, we check the brand name and if it 
matches any of our patterns, and if so, we print out the factory's 
location. 


If the brand name does not match any of our case statements, we print 
out a default message: an unknown car brand. 
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Conclusion 


| would advise you to try and modify the script and play with it a bit so 
that you could practice what you've just learned in the last two 
chapters! 


For more examples of Bash case statements, make sure to check 
chapter 16, where we would create an interactive menu in Bash using a 
cases Statement to process the user input. 
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Bash Loops 


As with any other language, loops are very convenient. With Bash you 
can use for loops, while loops, and until loops. 
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For loops 


Here is the structure of a for loop: 


for var in ${list} 
do 

your _commands 
done 


Example: 


#!/bin/bash 
users="devdojo, bobby, tony" 


for user in ${users} 
do 

echo "${user}" 
done 


A quick rundown of the example: 


¢ First, we specify a list of users and store the value in a variable 
called $users. 

¢ After that, we start our for loop with the for keyword 

¢ Then we define a new variable which would represent each item 
from the list that we give. In our case, we define a variable called 
user, which would represent each user from the $users variable. 

¢ Then we specify the in keyword followed by our list that we will 
Kote) om dal gelerela 

¢ On the next line, we use the do keyword, which indicates what we 
will do for each iteration of the loop 

¢ Then we specify the commands that we want to run 

¢ Finally, we close the loop with the done keyword 
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You can also use for to process a Series of numbers. For example here 
is one way to loop through from 1 to 10: 


#!/bin/bash 


Tor Mum intl, LO} 
do 

echo ${num} 
done 
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While loops 


The structure of a while loop is quite similar to the for loop: 


while [ your condition ] 
do 

your_conditions 
done 


Here is an example of a while loop: 


#!/bin/bash 


counter=1 
while [[ $counter -le 10 ]] 
do 
echo $counter 
((counter++) ) 
done 


First, we specified a counter variable and set it to 1, then inside the 
loop, we added counter by using this statement here: ((counter++) ). 
That way, we make sure that the loop will run 10 times only and would 
not run forever. The loop will complete as soon as the counter becomes 
10, as this is what we've set as the condition: while [[ $counter -le 
Ce 


Let's create a script that asks the user for their name and not allow an 
empty input: 
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#!/bin/bash 
read -p "What is your name? " name 


while [[ -z ${name} ]] 


do 

echo "Your name can not be blank. Please enter a valid 
name!" 

read -p "Enter your name again? " name 
done 


echo "Hi there ${name}" 


Now, if you run the above and just press enter without providing input, 
the loop would run again and ask you for your name again and again 
until you actually provide some input. 
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Until Loops 


The difference between until and while loops is that the until loop 
will run the commands within the loop until the condition becomes true. 


S)aae (nel aoe 


until [ your condition ] 
do 

your _commands 
done 


Example: 


#!/bin/bash 


count=1 
until [ $count -gt 10 ] 
do 
echo $count 
((count++) ) 
done 
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Continue and Break 


As with other languages, you can use continue and break with your 
bash scripts as well: 


* continue tells your bash script to stop the current iteration of the 
loop and start the next iteration. 
¢ break tells your bash script to end the loop straight away. 
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Bash Functions 


Functions are a great way to reuse code. The structure of a function in 
bash is quite similar to most languages: 


function function name() { 
your _commands 


i 


You can also omit the function keyword at the beginning, which would 
I SOR Ole 


function name() { 
your_commands 


} 


| prefer putting it there for better readability. But it is a matter of 
personal preference. 


Example of a "Hello World!" function: 


#!/bin/bash 
function hello(){ 


echo "Hello World Function!" 


i 


hello 
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Notice: One thing to keep in mind is that you should not add the 
ote] a=aa Mal =s]SM NL AL=1p IYO Le mere) Ua om el aledlo) an 


Passing arguments to a function work in the same way as passing 
arguments to a Script: 

#!/bin/bash 

function hello(){ 


echo "Hello $1!" 
i; 


hello DevDojo 


In the next few chapters we will be using functions a lot! 
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Debugging, testing and 
shortcuts 


In order to debug your bash scripts, you can use -x when executing 
your scripts: 


bash -x ./your_script.sh 


Or you can add set -x before the specific line that you want to debug, 
set -x enables a mode of the shell where all executed commands are 
printed to the terminal. 


Another way to test your scripts is to use this fantastic tool here: 


https://www.shellcheck.net/ 


Just copy and paste your code into the textbox, and the tool will give 
you some suggestions on how you can improve your script. 


You can also run the tool directly in your terminal: 
https://github.com/koalaman/shellcheck 
If you like the tool, make sure to star it on GitHub and contribute! 


As a SysAdmin/DevOps, | spend a lot of my day in the terminal. Here 
are my favorite shortcuts that help me do tasks quicker while writing 
Bash scripts or just while working in the terminal. 


The below two are particularly useful if you have a very long command. 
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¢ Delete everything from the cursor to the end of the line: 


Ctrl + k 


¢ Delete everything from the cursor to the start of the line: 


Ctrl + u 


e Delete one worked backward from cursor: 


Ctrl + w 


¢ Search your history backward. This is probably the one that | use 
the most. It is really handy and speeds up my work-flow a lot: 


(Cia Ge oe 


¢ Clear the screen, | use this instead of typing the clear command: 


a a al Bl 


¢ Stops the output to the screen: 


Ctrl +s 


¢ Enable the output to the screen in case that previously stopped by 
Ctrl + s: 


od 





| Ctrl +q 


e Terminate the current command 


| Ctrl +c 


¢ Throw the current command to background: 


| Ctrl + z 


| use those regularly every day, and it saves me a lot of time. 


If you think that I've missed any feel free to join the discussion on the 
DigitalOcean community forum! 
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Creating custom bash 
commands 


As a developer or system administrator, you might have to spend a lot 
of time in your terminal. | always try to look for ways to optimize any 
repetitive tasks. 


One way to do that is to either write short bash scripts or create custom 
commands also known as aliases. For example, rather than typing a 
really long command every time you could just create a short cut for it. 


516: 





chives 


Let's start with the following scenario, as a system admin, you might 
have to check the connections to your web server quite often, so | will 
use the netstat command as an example. 


What | would usually do when | access a Server that is having issues 
with the connections to port 80 or 443 is to check if there are any 
services listening on those ports and the number of connections to the 
ele) aace 


The following netstat command would show us how many TCP 
connections on port 80 and 443 we currently have: 


netstat -plant | grep '80\|443' | grep -v LISTEN | wc -l 


This is quite a lengthy command so typing it every time might be time- 
consuming in the long run especially when you want to get that 
information quickly. 


To avoid that, we can create an alias, so rather than typing the whole 
command, we could just type a short command instead. For example, 
lets say that we wanted to be able to type conn (short for connections) 
and get the same information. All we need to do in this case is to run 
the following command: 


alias conn="netstat -plant | grep '80\|443' | grep -v LISTEN | 
we -L" 


That way we are creating an alias called conn which would essentially 
be a ‘short cut' for our long netstat command. Now if you run just 
conn: 
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conn 


You would get the same output as the long netstat command. You can 
get even more creative and add some info messages like this one here: 


alias conn="echo ‘Total connections on port 80 and 443:' ; 
netstat -plant | grep '80\|443' | grep -v LISTEN | wc -l" 


Now if you run conn you would get the following output: 


Total connections on port 80 and 443: 
a2 


Now if you log out and log back in, your alias would be lost. In the next 
step you will see how to make this persistent. 
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Making the change persistent 


In order to make the change persistent, we need to add the alias 
command in our shell profile file. 


By default on Ubuntu this would be the ~/.bashrc file, for other 
operating systems this might be the ~/.bash_ profte. With your 
favorite text editor open the file: 


nano ~/.bashrc 


Go to the bottom and add the following: 


alias conn="echo ‘Total connections on port 80 and 443:' ; 
netstat -plant | grep '80\|443' | grep -v LISTEN | wc -l" 


Save and then exit. 


That way now even if you log out and log back in again your change 
would be persisted and you would be able to run your custom bash 
command. 
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Listing all of the available aliases 


To list all of the available aliases for your current shell, you have to just 
run the following command: 


alias 


This would be heady in case that you are seeing some weird behavior 
with some commands. 
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Conclusion 


This is one way of creating custom bash commands or bash aliases. 


Of course, you could actually write a bash script and add the script 
inside your /usr/bin folder, but this would not work if you don't have 
root or sudo access, whereas with aliases you can do it without the 
need of root access. 


Notice: This was initially posted on DevDojo.com 
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Write your first Bash script 


Let's try to put together what we've learned so far and create our first 
Bash script! 
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Planning the script 


As an example, we will write a script that would gather some useful 
information about our server like: 


¢ Current Disk usage 
¢ Current CPU usage 
¢ Current RAM usage 
¢ Check the exact Kernel version 


Feel free to adjust the script by adding or removing functionality so that 
it matches your needs. 


60 





Writing the script 


The first thing that you need to do is to create a new file with a .sh 
extension. | will create a file called status.sh as the script that we will 
create would give us the status of our server. 


Once you've created the file, open it with your favorite text editor. 


As we've learned in chapter 1, on the very first line of our Bash script 
we need to specify the so-called Shebang: 


#!/bin/bash 


All that the shebang does is to instruct the operating system to run the 
Script with the /bin/bash executable. 
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Adding comments 


Next, as discussed in chapter 6, let's start by adding some comments 
so that people could easily figure out what the script is used for. To do 
that right after the shebang you can just add the following: 


#!/bin/bash 


# Script that returns the current server status 


oy 





PUCCIO aie mC lars lets 


Then let's go ahead and apply what we've learned in chapter 4 and add 
some variables which we might want to use throughout the script. 


To assign a value to a variable in bash, you just have to use the = sign. 
For example, let's store the hostname of our server in a variable so that 
we could use it later: 


server _name=$(hostname) 


By using $() we tell bash to actually interpret the command and then 
assign the value to our variable. 


Now if we were to echo out the variable we would see the current 
hostname: 


echo $server_ name 
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Adding your first function 


As you already know after reading chapter 12, in order to create a 
function in bash you need to use the following structure: 


function function name() { 
your _commands 


i 


Let's create a function that returns the current memory usage on our 
server: 


function memory check() { 
echo "" 
echo "The current memory usage on ${server_name} is: " 
free -h 
echo "" 


Quick run down of the function: 


¢ function memory check() { - this is how we define the function 

*echo "" - here we just print a new line 

eecho "The current memory usage on ${server name} is: "- 
here we print all a small message and the $server name variable 

¢ } - finally this is how we close the function 


Then once the function has been defined, in order to call it, just use the 
name of the function: 


oy 





# Define the function 
function memory check() { 


echo uu 
echo "The current memory usage on ${server_ name} is: " 
free -h 
echo tT 


i 


# Call the function 
memory check 
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Adding more functions challenge 


Before checking out the solution, | would challenge you to use the 
function from above and write a few functions by yourself. 


The functions should do the following: 


¢ Current Disk usage 
¢ Current CPU usage 
¢ Current RAM usage 
¢ Check the exact Kernel version 


Feel free to use google if you are not sure what commands you need to 
use in order to get that information. 


Once you are ready, feel free to scroll down and check how we've done 
it and compare the results! 


Note that there are multiple correct ways of doing it! 
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The sample script 


Here's what the end result would look like: 


#!/bin/bash 


Bd 

# BASH script that checks: 

5 - Memory usage 

Fd - CPU load 

4 - Number of TCP connections 
# - Kernel version 

cd 


server _name=$ (hostname) 


function memory check() { 


echo "" 
echo "Memory usage on ${server_name} is: " 
sacl oe 
echo "" 
} 
function cpu _check() { 
echo "" 
echo "CPU load on ${server name} is: " 
echo "" 
uptime 
echo "" 
i 
function tcp check() { 
echo "" 
echo "TCP connections on ${server_ name}: " 
echo "" 
cat /proc/net/tcp | we -l 
echo "" 
} 
function kernel_check() { 
echo "" 
echo "Kernel version on ${server name} is: " 
echo "" 


ey! 





uname -fr 
echo te 


i 


function all _checks() { 
memory check 


cpu_check 
tcp check 
kernel_check 
i 
all_checks 
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Conclusion 


Bash scripting is awesome! No matter if you are a DevOps/SysOps 
engineer, developer, or just a Linux enthusiast, you can use Bash 
Scripts to combine different Linux commands and automate boring and 
repetitive daily tasks, so that you can focus on more productive and fun 
things! 


Notice: This was initially posted on DevDojo.com 
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Creating an interactive menu 
in Bash 


In this tutorial, | will show you how to create a multiple-choice menu in 
Bash so that your users could choose between what action should be 
=> Te] K oleh 


We would reuse some of the code from the previous chapter, so if you 
have not read it yet make sure to do so. 
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Planning the functionality 


Let's start again by going over the main functionality of the script: 


¢ Checks the current Disk usage 
¢ Checks the current CPU usage 
¢ Checks the current RAM usage 
¢ Checks the check the exact Kernel version 


In case that you don't have it on hand, here is the script itself: 


#!/bin/bash 


Bd 

# BASH menu script that checks: 
F - Memory usage 

4 - CPU load 

4 - Number of TCP connections 
Fa - Kernel version 

Fd 


server _name=$(hostname) 


function memory _check() { 
echo "" 
echo "Memory usage on ${server_ name} is: 
free -h 
echo "" 


i 


function cpu _check() { 
echo "" 
echo "CPU load on ${server name} is: " 
echo "" 
uptime 
echo "" 


i 


function tcp check() { 
echo te 
echo "TCP connections on ${server_ name}: 


ql 





echo tL 
cat /proc/net/tcp | we -l 


echo "" 
} 
function kernel_check() { 
echo "" 
echo "Kernel version on ${server name} is: 
echo "" 
uname -r 
echo "" 


i 


function all_checks() { 
NiT=11Ce) AVAL AL =10N.4 
cpu_check 
tcp check 
kernel_check 


We will then build a menu that allows the user to choose which function 


to be executed. 


Of course, you can adjust the function or add new ones depending on 


your needs. 
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Adding some colors 


In order to make the menu a bit more 'readable' and easy to grasp at 
first glance, we will add some color functions. 


At the beginning of your script add the following color functions: 


cd 

# Color Variables 
Bd 

green='\e[32m' 

OU (- MN Boro) 
clear='\e[0Om' 


ia 
# Color Functions 
cia 
ColorGreen() { 
echo -ne $green$l$clear 
} 
ColorBlue(){ 
echo -ne $blue$1$clear 
} 


You can use the color functions as follows: 


echo -ne $(ColorBlue 'Some text here') 


The above would output the Some text here string and it would be 
oUt 
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Adding the menu 


Finally, to add our menu, we will create a separate function with a case 
Switch for our menu options: 


iialen et 
echo -ne 
My First Menu 
$(ColorGreen '1)') Memory usage 
$(ColorGreen '2)') CPU load 
ColorGreen '3)') Number of TCP connections 

a) 

) 


( 
ca 
$(ColorGreen Kernel version 
$(ColorGreen '5)') Check ALL 
$(ColorGreen '0)') Exit 
$(ColorBlue ‘Choose an option:') " 
read a 
case $a in 
1) memory check ; menu ;; 
2) cpu_check ; menu ;; 
3) tcp check ; menu ;; 
4) kernel_check ; menu ;; 
5) all_checks ; menu ;; 
0) exit 0 ;; 
*) echo -e $red"Wrong option. "$clear; 
WrongCommand; ; 
Sis¥o1e 


i 


A quick rundown of the code 


First we just echo out the menu optsions with some color: 
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echo -ne 

My First Menu 

$(ColorGreen '1)') Memory usage 

)') CPU load 

ColorGreen '3)') Number of TCP connections 
)') Kernel version 

ColorGreen '5)') Check ALL 

ColorGreen '0)') Exit 

ColorBlue ‘Choose an option:') 


Then we read the answer of the user and store it in a variable called $a: 


read a 


Finally, we have a switch case which triggers a different function 
depending on the value of $a: 


case $a in 
1) memory check ; menu ;; 
2) cpu_check ; menu ;; 
3) tcp check ; menu ;; 
4) kernel_check ; menu ;; 
5) all_checks ; menu ;; 
0) exit 0 ;; 
*) echo -e $red"Wrong option. "$clear; 
WrongCommand; ; 
Sito e 


At the end we need to call the menu function to actually print out the 
menu: 


# Call the menu function 
menu 


ie) 





KATO AICo ates 


In the end, your script will look like this: 


#!/bin/bash 


Bd 

# BASH menu script that checks: 
5 - Memory usage 

FJ - CPU load 

4 - Number of TCP connections 
# - Kernel version 

cd 


server _name=$ (hostname) 


function memory check() { 


echo "" 
echo "Memory usage on ${server_name} is: " 
sacl oe 
echo "" 
} 
function cpu _check() { 
echo "" 
echo "CPU load on ${server name} is: " 
echo "" 
uptime 
echo "" 
i 
function tcp _check() { 
echo "" 
echo "TCP connections on ${server_ name}: " 
echo "" 
cat /proc/net/tcp | we -l 
echo "" 
} 
function kernel_check() { 
echo "" 
echo "Kernel version on ${server_ name} is: " 
echo "" 
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uname -r 
echo "" 


i 


function all _checks() { 
memory check 


cpu_check 
tcp check 
kernel_check 
i 
Sg 
# Color Variables 
Sa 


green='\e[32m' 
blue='\e[34m' 
clear='\e[0Om' 


bz 
# Color Functions 
bz 


ColorGreen() { 
echo -ne $green$l$clear 


: 
ColorBlue(){ 

echo -ne $blue$1$clear 
} 


ii=alen nt 
echo -ne 
My First Menu 
$(ColorGreen '1)') Memory usage 
$(ColorGreen '2)') CPU load 
ColorGreen '3)') Number of TCP connections 
4)') 
ye, 


ColorGreen ' Kernel version 
ColorGreen ' Check ALL 
ColorGreen '0)') Exit 
ColorBlue ‘Choose an option:') " 
iact- le mare) 
case $a in 
1) memory check ; menu ;; 
2) cpu_check ; menu ;; 
3) tcp check ; menu ;; 
4) kernel_check ; menu ;; 
5) all_checks ; menu ;; 


\ 
$ ( 
Ti 
oT 
$ ( 
$ ( 


Val 





0) exit 0 ;; 
*) echo -e $red"Wrong option. "$clear; 
WrongCommand; ; 
SE¥o16 


i 


# Call the menu function 
menu 


To test the script, create a new filed with a .sh extension, for example: 
menu.sh and then run it: 


bash menu.sh 


The output that you would get will look like this: 


My First Menu 

1) Memory usage 

2) CPU load 

3) Number of TCP connections 
4) Kernel version 

5) Check ALL 

0) Exit 

Choose an option: 


You will be able to choose a different option from the list and each 
number will call a different function from the script: 
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My First Menu 

SOO =e) avery =<) 

2) CPU load 

3) Number of TCP connections 
4) Kernel version 

5) Check All 

Cy asks 

Choose an option: 1 


Memory usage on bobbyiliev is: 


total used a) shared buff/cache 
ue Ie Ube mse 85Mi oi be mal ae 
Swap: ai Cr 336Mi SPUR 
My First Menu 
SDL slavery =<) 
2) CPU load 


3) Number of TCP connections 
4) Kernel version 

5) Check All 

Casts 

Choose an option: i 
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Conclusion 


You now know how to create a Bash menu and implement it in your 
scripts so that users could select different values! 


Notice: This content was initially posted on DevDojo.com 
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Executing BASH scripts on 
Multiple Remote Servers 


Any command that you can run from the command line can be used in 
a bash script. Scripts are used to run a series of commands. Bash is 
available by default on Linux and macOS operating systems. 


Let's have a hypothetical scenario where you need to execute a BASH 
Script on multiple remote servers, but you don't want to manually copy 
the script to each server, then again login to each server individually 
and only then execute the script. 


Of course you could use a tool like Ansible but lets learn how to do that 
with Bash! 
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Prerequisites 


For this example | will use 3 remote Ubuntu servers deployed on 
DigitalOcean. If you don't have a Digital Ocean account yet, you can 
Sign up for DigitalOcean and get $100 free credit via this referral link 
aolace 


https://m.do.co/c/2a9bba940f39 


Once you have your Digital Ocean account ready go ahead and deploy 
3 droplets. 


I've gone ahead and created 3 Ubuntu servers: 


82 





Learning 


Class project / Educational purposes 


Resources 


DROPLETS (3) 


e © devoot 


e © devood3 


e © dev0od2 





I'll put a those servers IP's in a servers.txt file which | would use to 
loop though with our Bash script. 


If you are new to DigitalOcean you can follow the steps on how to 
create a Droplet here: 


¢ How to Create a Droplet from the DigitalOcean Control Panel 


You can also follow the steps from this video here on how to do your 
initial server setul: 
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¢ How to do your Initial Server Setup with Ubuntu 


Or even better, you can follow this article here on how to automate your 
initial server setup with Bash: 


Automating Initial Server Setup with Ubuntu 18.04 with Bash 


With the 3 new servers in place, we can go ahead and focus on running 
our Bash script on all of them with a single command! 
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The BASH Script 


| will reuse the demo script from the previous chapter with some slight 
changes. It simply executes a few checks like the current memory 
usage, the current CPU usage, the number of TCP connections and the 
version of the kernel. 


#!/bin/bash 


cad 

# BASH script that checks the following: 
5 - Memory usage 

#  - CPU load 

# - Number of TCP connections 

ca - Kernel version 

cai 

cad 

# Memory check 

## 


server _name=$(hostname) 


function memory _check() { 
echo "#######" 
echo "The current memory usage on ${server_name} is: " 
free -h 
echo "#######" 


function cpu _check() { 
echo "#######" 
echo "The current CPU load on ${server_ name} is: " 
echo "" 
uptime 
echo "#######" 
: 


function tcp check() { 
echo "#######" 
echo "Total TCP connections on ${server name}: " 
echo Ta 
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cat /proc/net/tcp | we -l 
eee ec ae 
i 


function kernel_check() { 
echo "#######" 
echo "The exact Kernel version on ${server_name} is: " 
echo "" 
uname -r 
echo "#######" 
i 


function all _checks() { 
NiT=t1Te) AVAL AL =10N.4 


cpu_check 
tcp check 
kernel_check 
} 
all_checks 


Copy the code bellow and add this in a file called remote _check.sh. You 
can also get the script from here. 
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Running the Script on all Servers 


Now that we have the script and the servers ready and that we've 
added those servers in our servers.txt file we can run the following 
command to loop though all servers and execute the script remotely 
without having to copy the script to each server and individually 
connect to each server. 


for server in $(cat servers.txt) ; do ssh your _user@${server} 
‘bash -s' < ./remote check.sh ; done 


What this for loop does is, it goes through each server in the servers.txt 
file and then it runs the following command for each item in the list: 


ssh your_user@the server ip ‘bash -s' < ./remote_check.sh 


You would get the following output: 


ow 





biliev@bobby-mac:~/devdojo$ for server in $Ccat 


ete e sie a id 
IMU aa etn ou ela ARP cle( ies CH 

beh el uséd baat) 
Se 985M 96M 679M 
ATiVeloh QB QB 33 
Dee ead 
eee ie ad 


The current CPU load on devQQ1 is: 


16:08:17 up 7 min, 
Sad 
eee ad 
Total TCP connections on dev0Q1: 


@ users, load average 


ry 
See eed 
Sed 
The exact Kernel version on dev@Q1 is: 


4.15.0-52-generic 


Sed 

Dad 

Bistro aaa am =U A x (e(- Cee) 
total yar baat) 

A be} 35)0 be 683M 

Aye] os QB QB QB 

HY 

Sad 


The current CPU load on devQ@3 is: 


16:08:20 up 7 min, 
Sid 
See e read 
Total TCP connections on dev0Q3: 


@ users, load average 


ry 
Deere iad 
Decca 
The exact Kernel version on devQQ3 is: 


4.15.0-52-generic 
aaa eed 







shared buf£/cache 
BTL 'Q9M 


an 


shared buff/cache 
BTL rab) 


: 0.00, 0.05, 0.04 
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; do ssh root@${server} "bash -s' 





./remote_check.sh 


Our Bash Script 





available 
752M 


List with server IPs 


available 
757M 





Conclusion 


This is just a really simple example on how to execute a simple script 
on multiple servers without having to copy the script to each server and 
without having to access the servers individually. 


Of course you could run a much more complex script and on many 
more servers. 


If you are interested in automation, | would recommend checking out 
the Ansible resources page on the DigitalOcean website: 


Ansible Resources 


Notice: This content was initially posted on DevDojo 
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Work with JSON in BASH 
using jq 


The jq command-line tool is is a lightweight and flexible command-line 
JSON processor. It is great for parsing JSON output in BASH. 


One of the great things about jq Is that it is written in portable C, and it 
has zero runtime dependencies. All you need to do is to download a 
Single binary or use a package manager like apt and install it with a 
Single command. 
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Planning the script 

For the demo in this tutorial, | would use an external REST API that 
returns a simple JSON ouput called the QuizAPI: 

https://quizapi.1o/ 

If you want to follow along make sure to get a free API key here: 


https://quizapi.io/clientarea/settings/token 





The QuizAPI is free for developers. 


on 





Installing jq 


There are many ways to install jg on your system. One of the most 
Straight forward ways to do So is to use the package manager 
depending on your OS. 


Here is a list of the commands that you would need to use depending 
on your OS: 


¢ Install jq on Ubuntu/Debian: 


| sudo apt-get install jq 


¢ Install jq on Fedora: 


| sudo dnf install jq 


¢ Install jq on openSUSE: 


| sudo zypper install jq 


¢ Install jq on Arch: 


| sudo pacman -S jq 


¢ Installing on Mac with Homebrew: 


| brew install jq 


e Install on Mac with MacPort: 


4 





port install jq 


If you are using other OS, | would recommend taking a look at the 
official documentation here for more information: 


https://stedolan.github.io/jg/download/ 


Once you have jq installed you can check your current version by 
running this command: 


Sie ea 1 Pele 
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Parsing JSON with jq 


Once you have jq installed and your QuizAPI API Key, you can parse the 
JSON output of the QuizAPI directly in your terminal. 


First, create a variable that stores your API Key: 


API_KEY=YOUR API KEY HERE 


In order to get some output from one of the endpoints of the QuizAPI 
you can use the curl command: 


elite 
"https://quizapi.io/api/v1/questions?apikey=${API KEY}&limit=1 
(Oa 


For a more specific output, you can use the QuizAPI URL Generator 
a =1ace 


https://quizapi.io/api-config 


After running the curl command, the output which you would get would 
look like this: 








Tite: 
PODS net) erate ee ees ec eee ees eee ce tee ee eee 
pee ears UR st Me Ree eg Aes eee etn se te Get eee Rae Cee ee Meee SMS ee cus tesco eee eet ete Dee Gree eas een 
<em>", "answer _c":"<italico","enswer_d":"<it>","answer_e":null, "answer_f":nul]},"multiple_correct_answers":"false”,”: bz ‘answer_a_correct":"false”,"answer_b_correct":"true”,"answer_c_correct":"false","answer_d_correct™:"f 
alse", "answer_e correct” :"false” ,”answer_f_correct”:"false"}, "correct_answer” :"answer_b”,"explanation” :nul1, “tip” :null, “tags”: [{"name” :"HIML"}], “category” :"", “difficulty” :"Easy”}, {"id" :783, “question” :"What does the fopen() function req 
uires for it to execute properly:”,"description”:null,"answers":{"answer_a":"It requires two arguments stating first the file name and then mode in which to operate.","answer_b":"It requires one argument, the file name.","answer_c":"It 
requires one argument, the file name. With a second argument being available if we want to use it, in which mode to oparate.”,"answer_d":"It requires two arguments, the file name and the extension of the file we are opening.’ 
e":nul, "enswer_#":null},"multiple_correct_answers":"false”,"correct_answers":{"answer_a_correct™:"true”, "answer_b_correct":"false","answer_c correct": "false", "answer_d_correct”:"false”,"answer_e_correct":"false”,"answer_f_correct":"fa 


iard"}, {"id" :827, "question": "Docker containers are based on open standard”, "description" :null, “answers” :{"answer_a”:"Allo 








Pere ae Te Ee a 


n Pa ase ae eee : 
"true", "answer_c_correct”:"false","answer_d_correct":"false”,"answer_e_correct”:"false","answer_f_correct":"false"},"correct_answ 























BE ea aa ee Ee Mt ea we eas ard Oa ea ed Peso MRK Cala) 

















"false", "answer_d_correct":"false”," 
":"How can you make a numbered list 








reget Seas a tt eed 
teen me eRe esto Cesc 










"null, "tags":[{"name" :"MySQL"}], "category": "Linux", "difficulty": "Easy"}, ("i 
nsner_c":"strlen","answer_d”:"strcount™,"ansner_e":null,"answer_#":null}, 
alse” ,"answer_e_correct":"false","answer_f_correct”:"false"},"correct_answer” 
er container immediately?" , "description" :null,"answers":{"answer_a":"$ docker terminate”, " 








ety Om trea ares eeie a ae ee va be E 
Fax Bee ster eee an eG oa Ren re ae aetna ert ae 













nsuer_a", "explanation 
ee a ee ce me ete ee 
ar ee ee a 
eee es ee ae ee cee ae ne aera eerste ee Aste te ee re ere eet tsi 
the status of a Docker container.","answer_b":"The command \u291cdocker ps status\u2@id is used for identifying the status of a Docker container.","answer_c":"The command \u2@1cdocker ps -p\u2@id is used for identifying the status of a 
Docker container.”,"answer_d”:null,"answer_e":null,"answer_f":null},"multiple_correct_answers":"false","correct_answers”:{"answer_a_correct":"true","answer_b_correct":"false","answer_c_correct":"false”,"answer_d_correct":"false”,"answ 
er_e_correct":"false","answer_f_correct’ 

on\"?", "description" :null, "answers" : eas Owe ee Test nr sc oct Om ae ene ue eee OR ar ERR st CM ae aie CVE unc ea aero 
aries ts ss ce a eer ve Ne seta ake Oe ane ee chose scare Tne Meine ek Eg ee ears eRe a at mee TR et ure Tne nena Gr 












This could be quite hard to read, but thanks to the jq command-line 
tool, all we need to do is pipe the curl command to jg and we would see 


ye 





a nice formated JSON output: 


curl 
"https://quizapi.io/api/v1/questions?apikey=${API KEY}&limit=1 
0" | jq 


Note the | jq atthe end. 


In this case the output that you would get would look something like 
ima) 


[ 
ag 
"id": 475, 
“question": “How can you make A bulleted list with numbers?", 
i: (“rae t sje ir) n 


“answers": { 
“answer_a": "<dl>", 
“answer_b": "<ul>", 
“answer_c"s "<ol>", 
“answer_d"s "<list>", 


2 

“multiple_correct_answers": “false”, 

“correct_answers": { 
“answer_a_correct": "false", 


art ae ela lo a 
bari a ol on 
“answer_d_correct"s 


“false”, 
at a 
“false”, 


“answer_e_correct": “false”, 
“answer_f_correct": “false™ 


oa 
alee ale a a 
“explanation”: S 
oe , 
“tags": [ 
“name": "HTML" 

he 

u 


> 
“category": "Code", 
-bame tall a5 ar ie 1 a 


1b 

x 
eter... 
"question": "Suppose you have 3 containers running and out of these, you wish to access one of them. How do you access a running container?", 
“description": 4 


“answers": { 


“answer_b": "$ docker exec --it <container id>", 
“answer_c": "$ docker exec -it <container id> bash", 


Ely a ey “ 
El aay 5 
urs aie ae 


2 
“multiple_correct_answers": “false”, 
“correct_answers": { 
bari) oa on a 8 
“answer_b_correct": "false", 
“answer_c_correct": "true", 


Now, this looks much nicer! The jq command-line tool formatted the 
output for us and added some nice coloring! 
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Getting the first element with jq 


Let's say that we only wanted to get the first element from the JSON 
output, in order to do that we have to just specify the index that we 
want to see with the following syntax: 


jq .[0] 


Now, if we run the curl command again and pipe the output to jq .[0] 
like this: 


curl 
"https://quizapi.io/api/v1/questions?apikey=${API KEY}&limit=1 
| aicied eu 


You will only get the first element and the output will look like this: 


ag 
"id": 792, 
“question”: “What is Hypervisor?", 
“description”: ’ 


“answers": { 
ari ae I cao a od ae et Le a Nee s-lete ets 1s) tee a eee eee TOL sige S ML le iP ts Methane) eee 
“answer_b": "A hypervisor is a hardware that makes optimization possible. It is also called Virtual Machine Monitor.", 
“answer_c"s "A hypervisor is a software that makes optimization possible. It is also called Virtual Machine Monitor.", 


“answer_d"s ry 
ar ae 5 
are L= a 


oe 

“multiple _correct_answers": "false", 

“correct_answers": { 
“answer_a_correct": "true", 
“answer_b_correct": "false", 
“answer_c_correct": "false", 
“answer_d_correct": "false", 
“answer_e_correct": "false", 
“answer_f_correct": "false" 

} 

“correct_answer": "answer_a", 

“explanation": . 

ba oT» ba 5 

"tags": [ 
‘ "name": "Docker" 
be 

‘Fy 

"category": "Docker", 

"difficulty": "Easy" 

} 
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Getting a value only for specific key 


Sometimes you might want to get only the value of a specific key only, 
let's say in our example the QuizAPI returns a list of questions along 
with the answers, description and etc. but what if you wanted to get the 
Questions only without the additional information? 


This is going to be quite straight forward with jq, all you need to do is 
add the key after jg command, so it would look something like this: 


jq .[].question 


We have to add the .[] as the QuizAPI returns an array and by 
Specifying .[] we tell jq that we want to get the .question value for all 
of the elements in the array. 


The output that you would get would look like this: 


"To use ‘mysqldbcopy" which privileges are required on the source server?" 
“Each virtual machine includes the application, the necessary binaries and 
“The GROUP BY command cannot be used with aggregate functions together" 
“Which command can be used to make variables of shell to be made available 
"To program additional authentication logic besides available authenticato1 
"Tags and test that are not directly displayed on the page are written in | 
“What is Solaris?” 

"“Kubernetes cluster data is stored in which of the following?” 

“Which command can you use to re-execute a previous command?” 

"You can add a row using SQL in a database with which of the following?” 


As you can see we now only get the questions without the rest of the 
VEL Loise 


Si 





LOST emia sytem ale 


Let's go ahead and create a small bash script which should output the 
following information for us: 


* Get only the first question from the output 

¢ Get all of the answers for that question 

¢ Assign the answers to variables 

¢ Print the question and the answers 

¢ To do that I've put together the following script: 


Notice: make sure to change the API_KEY part with your actual 
QuizAPI key: 
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#!/bin/bash 


cad 

# Make an API call to QuizAPI and store the output in a 
variable 

ces 

LUNG Une TGC ae 
'https://quizapi.io/api/vl/questions?apikey=API KEY&limit=10' 
Vey Ale N Palen el) 


Se 

# Get only the first question 
Se 

output=$(echo $output | jq .[0]) 


Sa 

# Get the question 

Sd 

question=$(echo $output | jq .question) 


bz 
# Get the answers 
Ez 


answer _a=$ 
answer b=$ 
answer c=$ 
answer _ d=$ 


echo $output | jq .answers.answer a) 
echo $output | jq .answers.answer b) 
echo $output | jq .answers.answer c) 
echo $output | jq .answers.answer d) 


a nN nN 


Se 
# Output the question 
## 


echo " 
Question: ${question} 


) ${answer a} 
) ${answer_ b} 
eS 1010 en 
) 


A 
3] 
@ 
D) ${answer_ d} 


If you run the script you would get the following output: 


Je) 





Question: “Which of the answers listed below refers to a package management system most often used on Debian 


A) "kde" 
B) “apt” 
c) ovr 
ata + 


We can even go further by making this interactive so that we could 
actually choose the answer directly in our terminal. 


There is already a bash script that does this by using the QuizAPI and 
ack 


You can take a look at that script here: 


¢ https://github.com/QuizApi/QuizAPI-BASH/blob/master/quiz.sh 
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Conclusion 


The jq command-line tool is an amazing tool that gives you the power 
to work with JSON directly in your BASH terminal. 


That way you can easily interact with all kinds of different REST APIs 
with BASH. 


For more information, you could take a look at the official 
documentation here: 


¢ https://stedolan.github.io/jg/manual/ 


And for more information on the QuizAPI, you could take a look at the 
official documentation here: 


¢ https://quizapi.io/docs/1.0/overview 


Notice: This content was initially posted on DevDojo.com 
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Working with Cloudflare API 
with Bash 


| host all of my websites on DigitalOcean Droplets and | also use 
Cloudflare as my CDN provider. One of the benefits of using Cloudflare 
is that it reduces the overall traffic to your user and also hides your 
actual server IP address behind their CDN. 


My personal favorite Cloudflare feature is their free DDoS protection. It 
has saved my servers multiple times from different DDoS attacks. They 
have a cool API that you could use to enable and disable their DDoS 
protection easily. 


This chapter is going to be an exercie! | challange you to go ahead and 
write a short bash script that would enable and disable the Cloudflare 
DDoS protection for your server automatically if needed! 
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Prerequisites 


Before following this guide here, please set up your Cloudflare account 
and get your website ready. If you are not sure how to do that you can 
follow these steps here: Create a Cloudflare account and add a website. 


Once you have your Cloudflare account, make sure to obtain the 
following information: 


¢ A Cloudflare account 
¢ Cloudflare API key 
¢ Cloudflare Zone ID 


Also, Make sure curl is installed on your server: 


| curl --version 


If curl is not installed you need to run the following: 


¢ For RedHat/CentOs: 


| yum install curl 


¢ For Debian/Ubuntu 


| apt-get install curl 
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Challange - Script requirements 


The script needs to monitor the CPU usage on your server and if the 
CPU usage gets high based on the number vCPU it would enable the 
Cloudflare DDoS protection automatically via the Cloudflare API. 


The main features of the script should be: 


¢ Checks the script CPU load on the server 

¢ In case of a CPU Spike the script triggers an API call to Cloudflare 
and enables the DDoS protection feature for the specified zone 

¢ After the CPU load is back to normal the script would disable the 
"I'm under attack" option and set it back to normal 
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Example script 


| already have prepared a demo script which you could use as a 
reference. But | encourage you to try and write the script yourself first 
and only then take a look at my script! 


To download the script just run the following command: 


wget 
https://raw.githubusercontent.com/bobbyiliev/cloudflare-ddos-p 
rotection/main/protection.sh 


Open the script with your favorite text editor: 


nano protection.sh 


And update the following details with your Cloudflare details: 


CF CONE ID=YOUR CF ZONE ID 
CF EMAIL ADDRESS=YOUR CF EMAIL ADDRESS 
CF API KEY=YOUR CF API KEY 


After that make the script executable: 


| chmod +x ~/protection.sh 


Finally, set up 2 Cron jobs to run every 30 seconds. To edit your crontab 
run: 


| crontab -e 
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And add the following content: 


* * * * * /path-to-the-script/cloudflare/protection.sh 
* * * * * ( sleep 30 ; /path-to-the- 
script/cloudflare/protection.sh ) 


Note that you need to change the path to the script with the actual path 
where you've stored the script at. 


anelo 





Conclusion 


This is quite straight forward and budget solution, one of the downsides 
of the script is that if your server gets unresponsive due to an attack, 
the script might not be triggered at all. 


Of course, a better approach would be to use a monitoring system like 
Nagios and based on the statistics from the monitoring system then you 
can trigger the script, but this scirpt challange could be a good learning 
experience! 


Here is another great resource on how to use the Discord API and send 
notifications to your Discord Channel with a Bash script: 


How To Use Discord Webhooks to Get Notifications for Your Website 
Status on Ubuntu 18.04 





Notice: This content was initially posted on DevDojo 
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BASH Script parser to 
Summarize Your NGINX and 
Apache Access Logs 


One of the first things that | would usually do in case | notice a high CPU 
usage on some of my Linux servers would be to check the process list 
with either top or htop and in case that | notice a lot of Apache or Nginx 
process | would quickly check my access logs to determine what has 
caused or is causing the CPU spike on my server or to figure out if 
anything malicious is going on. 


Sometimes reading the logs could be quite intimidating as the log might 
be huge and going though it manually could take a lot of time. Also, the 
raw log format could be confusing for people with less experience. 


Just like the previous chapter, this chapter is going to be a challange! 
You need to write a short bash script that would summarize the whole 
access log for you without the need of installing any additional 
software. 
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Script requirements 


This BASH script needs to parse and summarize your access logs and 
provide you with very useful information like: 


¢ The 20 top pages with the most POST requests 
¢ The 20 top pages with the most GET requests 
¢ Top 20 IP addresses and their geo-location 
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Example script 


| already have prepared a demo script which you could use as a 
reference. But | encourage you to try and write the script yourself first 
and only then take a look at my script! 


In order to download the script, you can either clone the repository with 
the following command: 


git clone 
https://github.com/bobbyiliev/quick access logs summary.git 


Or run the following command which would download the script in your 
current directory: 


wget 
https://raw.githubusercontent.com/bobbyiliev/quick access logs 
_summary/master/spike check 


The script does not make any changes to your system, it only reads the 
content of your access log and summarizes it for you, however, once 
you've downloaded the file, make sure to review the content yourself. 
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Running the script 


All that you have to do once the script has been downloaded is to make 
it executable and run it. 


To do that run the following command to make the script executable: 


| chmod +x spike check 


Then run the script: 


| ./spike check /path/to/your/access log 


Make sure to change the path to the file with the actual path to your 
access log. For example if you are using Apache on an Ubuntu server, 
the exact command would look like this: 


| ./spike check /var/log/apache2/access.log 


If you are using Nginx the exact command would be almost the same, 
but with the path to the Nginx access log: 


| ./Spike check /var/log/nginx/access. log 
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Understanding the output 


Once you run the script, it might take a while depending on the size of 
the log. 


The output that you would see should look like this: 


summarizing log.. 
This might take a while depending on the size of the log 


Top 20 GET requests: 
GET /blog/feed 
GET /robots.txt 
GET /roots.php 
GET /wp-content/themes/nervagq/js/jquery.flexslider.min.js 
GET /6avision/wp-content/themes/Lucid/images/widget-grey-bullet.png 


Most Recent top 20 GET requests: 
GET /blog/feed 
GET /robots.txt 
GET /roots.php 
GET /wp-content/themes/nervag/js/jquery.flexslider.min.js 
GET /6avision/wp-content/themes/Lucid/images/widget-grey-bullet.png 


Top 20 POST requests for: 
16 POST /wp-cron.php 
16 POST /xmlrpc.php 


Most Recent top 20 POST requests: 
16 POST /wp-cron.php 
16 POST /xmlrpc.php 


Top 20 IP addresses that have been accessing your site: 
Do you want geo location check for the IPs? [yes/no] 
yes 

64 - 5.255.253.7 - Russian Federation 

32 - 5.255.253.6 - Russian Federation 





Essentially what we can tell in this case is that we've received 16 POST 
requests to our xmlIrpc.php file which is often used by attackers to try 
and exploit WordPress websites by using various username and 
password combinations. 


In this specific case, this was not a huge brute force attack, but it gives 
us an early indication and we can take action to prevent a larger attack 
in the future. 


We can also see that there were a couple of Russian IP addresses 


ally 





accessing our site, so in case that you do not expect any traffic from 
Russia, you might want to block those IP addresses as well. 


ages: 





Conclusion 


This is an example of a simple BASH script that allows you to quickly 
Summarize your access logs and determine if anything malicious is 
going on. 


Of course, you might want to also manually go through the logs as well 
but it is a good challange to try and automate this with Bash! 


Notice: This content was initially posted on DevDojo 
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Sending emails with Bash and 
SSMTP 


SSMTP is a tool that delivers emails from a computer or a server to a 
configured mail host. 


SSMTP is not an email server itself and does not receive emails or 
manage a queue. 


One of its primary uses is for forwarding automated email (like system 
alerts) off your machine and to an external email address. 
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Prerequisites 


You would need the following things in order to be able to complete this 
tutorial successfully: 


e Access to an Ubuntu 18.04 server as a non-root user with sudo 
privileges and an active firewall installed on your server. To set 


these up, please refer to our Initial Server Setup Guide for Ubuntu 
18.04 


¢ An SMTP server along with SMTP username and password, this 
would also work with Gmail's SMTP server, or you could set up your 
own SMTP server by following the steps from this tutorial on 
[https://www.digitalocean.com/community/tutorials/how-to-install-a 
nd-configure-postfix-as-a-send-only-smtp-server-on- 
ubuntu-16-04](How to Install and Configure Postfix as a Send-Only 
SMTP Server on Ubuntu 16.04) 
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Installing SSMTP 


In order to install SSMTP, you’ll need to first update your apt cache with: 


| sudo apt update 


Then run the following command to install SSMTP: 


| sudo apt install ssmtp 


Another thing that you would need to install is mailutils, to do that 
run the following command: 


| sudo apt install mailutils 
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Configuring SSMTP 


Now that you have ssmtp installed, in order to configure it to use your 
SMTP server when sending emails, you need to edit the SSMTP 
configuration file. 


Using your favourite text editor opent the /etc/ssmtp/ssmtp.conf file: 


sudo nano /etc/ssmtp/ssmtp. conf 


You need to incldue the your SMTP configuration: 


root=postmaster 

mailhub=<*>your_smtp_host.com<*>:587 
hostname=<*>your_hostname<*> 
AuthUser=<*>your_ gmail _username@your smtp host.com<*> 
AuthPass=<*>your_ gmail _password<*> 
FromLineOverride=YES 

UseSTARTTLS=YES 


Save the file and exit. 
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Sending emails with SSMTP 


Once your configuration is done, in order to send an email just run the 
following command: 


echo "<*>Here add your email body<*>" | mail -s "<*>Here 
specify your email subject<*>" 
<“>your recepient email@yourdomain.com<*> 


You can run this directly in your terminal or include it in your bash 
scripts. 
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Sending A File with SSMTP (optional) 


If you need to send files as attachments, you can use mpack. 


To install mpack run the following command: 


sudo apt install mpack 


Next, in order to send an email with a file attached, run the following 
command. 


mpack -s "<*>Your Subject here<*>" your file.zip 
<“>your recepient email@yourdomain.com<*> 


The above command would send an email to 
<“>your recepient email@yourdomain.com<*> with the 
<*>your_ file.zip<*> attached. 
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Conclusion 


SSMTP is a great and reliable way to implement SMTP email 
functionality directly in bash scripts. 


For more information about SSMTP | would recommend checking the 
official documentation here. 


Notice: This content was initially posted on the DigitalOcean 
community forum. 
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Wrap Up 


Congratulations! You have just completed the Bash basics guide! 
If you found this useful, be sure to star the project on GitHub! 


If you have any suggestions for improvements, make sure to contribute 
pull requests or open issues. 


In this introduction to Bash scripting book, we just covered the basics, 
but you still have enough under your belt to start wringing some 
awesome scripts and automating daily tasks! 


As a next step try writing your own script and share it with the world! 
This is the best way to learn any new programming or scripting 
language! 


In case that this book enspired you to write some cool Bash scripts, 
make sure to tweet about it and tag @bobbyiliev_ so that we could 
(ial =le, aa ieme leh m 


Congrats again on completing this book! 
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